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USER INTERFACE SUBSTITUTION 
BACKGROUND 

The invention relates to systems and methods of defining a user interface for 
computer program. 

Traditionally, graphical user interfaces have been employed to facilitate interaction of 
a user with software in a computer system. For example, typical objects in a GUI are icons 
presented on a background or "desktop", with the icons representing a set of functions, 
application programs, system resources, or the like. These icons may be selected to instruct 
the computer to perform particular functions. 

GUI objects are generally created as part of a static application program construct that 
describes how a particular GUI object appears and what function is performed when that GUI 
is selected. Many visually-based application builder tools, such as Visual Basic, available 
from Microsoft Corporation of Redmond, Washington, have a graphical user interface that 
includes a "toolbox", a "form window", and a "property sheet," among others. A toolbox 
typically contains icons that represent different classes of components, or software modules. 
During a UI (user interface) construction session, a form window is displayed for the user to 
compose an application UI. A component selected from the toolbox may be "dropped", or 
otherwise placed, within the form window. Placing a component means placing an icon or 
other visual representation of the component, within the form window. A property sheet 
displays properties, or attributes, which relate to the selected component. The properties can 
include, but are not limited to, information relating to the size, color, and name of the 



Attorney's Docket NoS^W4/28001 

Client No. P254 

selected component. Programmers then insert code for each object to handle events 
generated by a user during operation. 

After an application has been developed, a user generally has a limited ability to 
customize the user interface by modifying properties associated with certain GUI objects. In 
general, a property sheet displays a fixed set of properties that can be modified to customize, 
or configure, a given component. Hence, a user can select and modify different properties in 
the property sheet in order to customize a component. Simple components typically have 
fewer associated properties than larger components, and can therefore be readily customized 
by making selections in a property sheet. 

As computer usage becomes increasingly widespread, the desire for custom computer 
applications that are specific to the needs of a particular user is also increasing. The desire is 
driven in part by the need to customize the software to the user's needs rather than force the 
user to adjust to the software, which can be tedious, time-consuming and non-productive. 

SUMMARY OF THE INVENTION 

In one aspect, a method defines a user interface for a computer program after 
execution of the computer program has begun by: reading a function description of a first 
function to be provided by the user interface; reading an appearance description of a first 
appearance to be presented by the user interface; associating the function description and the 
appearance description on the fly at run time; and executing the user interface with the 
associated function and appearance. 

Implementations of the invention may include one or more of the following. The 
function description or the appearance description can be replaced during program execution. 
The method can include reading a map defining multiple functions to be provided by the user 
interface including the first function; reading a fashion defining all appearances to be 
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presented by the user interface including the first appearance; associating the map and the 
function on the fly at run time; and executing the user interface with the associated map and 
function. The map or fashion can be replaced during program execution. The map can 
specify that a subordinate part of the user interface is specified by a second map-fashion pair. 
5 Events can be received from the map component or the fashion component. Business logic 
associated with the respective component can be executed. The components can be stored in 
a database. 

In a second aspect, a method defines a user interface for a computer program by: 
associating a map component and a fashion component on the fly at run time to generate the 
user interface; and executing the user interface with the associated function and appearance. 

Implementation of the method can include one or more of the following. A resource 
bundle associated with the map component can be loaded. Sub-components of the user 
interface can be loaded and instantiated. The fashion component can be called to allocate a 
resource to each sub-component. Each sub-component can be instructed to present itself in 
the user-interface. Events from the map component or the fashion component can be 
received. 

Advantages of the invention include one or more of the following. The map 
(controller) and the fashion (view) need not be static and each can be changed (or can change 
itself) independently of the other during execution of a program. By changing the map, the 
20 logic of the user interface can change while the same fashion is used to provide continuity of 
presentation. By changing the fashion without changing the map, the logic of the user 
interface can be presented with radically different appearances. Further, to provide 

-3- 



J3 10 



Attorney's Docket No^^4/28001 

Client No. P254 

flexibility, the map can specify that a subordinate part of a user interface is to be constructed 
using a second map-fashion pair. 

The logic and appearance of a user interface can be built independently, making the 
UI presentation more accessible to non-programmers. The UI can be modified without re- 
5 compiling the application. Because of late binding, the details of the UI presentation are 
established at execution time for the application. Hence, user interface development can be 
decoupled from code development. Hence, a visual designer can design a user interface 
without incurring delays associated with involving a programmer. 

The UI specification is declarative rather than procedural. The declarative aspect is 
10 advantageous in that abstract, high-level specifications can be synthesized without a detailed 
procedural description of the required action, such as calling the system to allocate a 
particular resource, to get a layout object, to modify the layout object, and to attach the 
modified layout object to another layout object. 

The appearances can be presented using a graphical interface, or can be presented in a 
15 non- visual modality such as a voice user interface that can be used in speech driven 
£ computers, including telephones and voice activated computers. Moreover, much of the UI 

is exposed in a way that facilitates localization. Thus, it is easy to create a user interface for 
an international product in which customers in different see different text, images, and other 
user interface elements. Use of the invention allows a corporation or other group to 
20 implement a corporate appearance for its user interface across multiple products. 



i 1 ? 



BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagram showing a system with an exemplary user interface object. 
Fig. 2 is a diagram illustrating an exemplary user interface. 
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Fig. 3 is a diagram illustrating cascaded user interface objects. 

Fig. 4 is a flowchart illustrating a process for generating presentations based on the 
user interface objects. 

Fig. 5 is a flowchart illustrating the initialization of objects associated with a user 
interface. 

Fig. 6 is a flowchart illustrating the operation of the user interface initialized in Fig. 5. 

Fig. 7 is an exemplary representation of the specification of a user interface. 

Fig. 8 is a diagram illustrating an embodiment of a process to generate a user 
interface presentation. 

Fig. 9 is a flowchart illustrating a second process for generating presentations based 
on the user interface objects. 

Fig. 10 is a diagram of an operating environment for the process of Fig. 4 or Fig. 9. 

Fig. 1 1 is a diagram of an exemplary computer capable of executing the process of 
Fig. 4 or Fig. 9. 



DETAILED DESCRIPTION 

As shown in Fig. 1, a system 100 has a user interface (UI) object. The UI object 102 
can have a graphical modality such as a two-dimensional or three-dimensional graphical user 
interface, or can have a different modality, such as a voice user interface commonly used in 
telephone response system. 

The object 102 has a map component 104. The map component 104 contains 

instructions for handling and processing objects contained in the user interface. The map 

component 104 thus embodies logic associated with the user interface object 102. The map 

component 104 knows that certain UI resources are available to it and are defined in a 

-5- 
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resource bundle 106 with a list of UI objects and their characteristics. The resource bundle 
can be created by an application programmer or can be programmatically generated. The 
resource bundle contains, directly or by reference, the layout rule for each UI object and the 
requisite system call that displays and enable that UI object to generate an event based on a 
user input. 

The UI objects in the resource bundle 106 can include, for example, a text-box that 
provides an area to enter or display text. The UI object can be a checkbox, which displays a 
True/False or Yes/No option. The UI objects can include combo-box that combines a text 
box with a list box and allows a user to type in a selection or select an item from a drop-down 
list. The UI objects can include a button that carries out a command or action when a user 
chooses it. Yet another exemplary UI object can be a horizontal scrollbar and vertical 
scrollbar, which allow a user to add scroll bars to controls that do not automatically provide 
them. 

Resource bundles contain locale-specific objects. When the application software 
needs a locale-specific resource, a "String" for example, the program can load it from the 
resource bundle that is appropriate for the current user's locale. In this way, program code 
can be largely independent of the user's locale by isolating most, if not all, of the locale- 
specific information in resource bundles. Each related subclass of the resource bundle 
contains the same items, but the items have been translated for the locale represented by that 
resource bundle subclass. For example, both MyResources (English version) and 
MyResources_de (German version) may have a String that is used on a button for confirming 
operations. In MyResources the String may contain "OK" and in MyResources_de it may 
contain "Gut." 



Attorney's Docket No. /28001 
Client No. P254 

When the program needs a locale-specific object, it loads the resource bundle class 
using a u getBundle()" method. The resource bundle lookup searches for classes with various 
suffixes on the basis of (1) the desired locale and (2) the default locale (baseclass); the search 
order is from lower-level (more specific) to parent-level (less specific). The result of the 
lookup is a class, but that class may be backed by a property file on disk. If a lookup fails, 
getBundle() throws a missing resource exception. Such an exception will generally cause the 
map component 104 to rely on a default locale. 

The map component 104 contains logic to handle one or more events generated by the 
UI objects. An event has a life cycle that begins with the action or condition that initiates the 
event and ends with the final response by the event handler. Typically, a user action or 
condition associated with the event occurs, and an event object is updated to reflect the 
conditions of the event. The event fires to notify the logic in the map component 104. The 
map component logic executes an event handler associated with the event is called. The 
event handler carries out its actions and returns. The event is provided to the next element in 
the hierarchy, and the event handler for that element is called. For events that bubble, if there 
is no event handler bound to the source element generating the event, the event handler for 
the next element up the element's hierarchy is called. 

The user interface object 102 also contains a fashion component 108. The fashion 
component 108 embodies the presentation, including the layout of the object 102. The 
fashion component is a collection whose elements represent each loaded form in an 
application. The fashion component 108 also derives its knowledge from a resource bundle 
110. In one embodiment, the resource bundle 106 is the same as the resource bundle 110. 

The user interface object 102 also can include additional user interfaces 110, 112 and 
1 14. The UIs 1 10, 1 12, and 1 14 provide programmatical support for the user to navigate to 
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another presentation. In one embodiment, the UI 1 10, 1 12 or 1 14 can be a button which, 
when clicked, moves users backwards or forwards in the application. 

During initialization, the application associated with the user interface object 102 
loads the map component 104 and the fashion compon ent 108 before creating a presentation 
5 to the user.^ The map component 104. and the fashion component 108 need not be static. The 
map component 104 and the fashion component 108 can be replaced during execution. By 
changing the map component 104, the logic of the user interface can be altered while the 
same fashion component 108 can still be used to provide a continuity of presentation. 
Correspondingly, the fashion component 108 can be changed during execution to allow the 
10 logic of the user interface to be presented with a radically different appearance or even in 
£ some non-visual modality, such as the voice user interface. 

^ A variety of data components can be identified with one or more map components 

*! and one or more fashion components. The map component as well as the fashion component 

** can change under the control of their code or can be changed by other map and fashion 

15 components. Thus, the user interface object 102 of Fig. 1 allows for an arbitrary 
fl composition. Two parts of the application user interface can be built independently using 

different map components or fashion component pairs. Alternatively, one map component 
can specify that a subordinate part of the user interface be constructed by a second map 
component 104 and a second fashion component 108 pair as needed. 
20 Fig. 2 illustrates an exemplary user interface. The user interface of Fig. 2 includes a 

top horizontal portion 120 that is positioned above a left lower portion 122 and right lower 
position 124. Further, each of the interface portions 120, 122 and 124 in turn can reference 
one or more secondary user interfaces. 
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Fig. 3 shows an exemplary structure associated with the user interface embodiment of 
Fig. 2. A user interface object 128 includes a map component 130 and a fashion component 
132. The map component 130 and fashion component 132 each references one or more 
resource bundles that specify the UI objects such as buttons and text-boxes, for example. 
5 Moreover, the user interface 128 references a top user interface object 134, a right user 
interface object 136 and a left user interface object 138. During execution, the logic 
associated with the map component 130 finds the required subcomponents. In this case, the 
logic finds three objects 134, 136 and 138. The logic instantiates each object using the name 
of the map and the fashion. The user can navigate down the user interface hierarchy for 

□ 10 subsequent display screens by selecting references.to dependent user interface objects using 

+ : various UI objects. 

, ^ The exemplary structure of Fig. 3 also shows details associated with the left user 

j«i interface object 138. Similar to the parent user interface object 128, the left user interface 

i^L object 138 can designate a plurality of additional user interface objects 140, 142, 144, and 

M* 15 146. Although Fig. 3 only shows UI objects associated with the left user interface objects 

'Hi 

l S 138, it is to be understood that similar objects exist for the top and right user interface objects 

134 and 136. Generally, the UI is developed through a process of recursive refinement until 
primitive UI components ( a button or a text label, for instance) are specified. The resulting 
structure is a rooted, directed graph. Also, the same map or fashion may appear at different 
20 levels of the UI composition. More particularly, for two component UI objects (each of class 
UI) with map and fashion objects, the two maps (fashions) are different objects but may be 
the same class. For example, in the actual application, certain classes for simple navigation 
and for providing a particular look and feel for conventional UI widgets are used repeatedly. 
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Turning now to Fig. 4, a process associated with loading and using the user interface 
is shown. First, the map component and a fashion component are loaded (step 202). Next, a 
user interface is generated based on the map component and the fashion component loaded in 
step 202. A user interface or presentation is then presented to the user (step 204). Next, the 
5 application determines whether a new presentation is required (step 206). If so, the map and 
fashion components are updated (step 208) and the process 200 loops back to step 204 to 
present Ihe user interface to the user. Alternatively, if a new presentation is not required, the 
process 200 executes the logic of the application (step 210). 

After executing the business logic, the process 200 determines whether additional 

10 presentations are required (step 212). If so, the process 200 loops back to step 208 to 

reference new map and fashion components (step 208). Alternatively, if a new presentation 
is not required in step 212, the process 200 determines whether the user wishes to exit the 
application (step 214). If not, the process loops back to step 210 to continue executing the 
application. From step 214, if the user wishes to exit, the process 200 exits. 

15 Fig. 5 shows a process 230 for initializing a user interface. First, the process loads 

one or more resource bundles associated with the map component or with a fashion 
component. Next, each subcomponent of the user interface is located (step 234). The 
located subcomponents are then instantiated (step 236). Next, the process 230 calls the 
fashion component to allocate resources to the subcomponents (step 238). Finally, each 

20 component is then instructed to present itself on the screen as a user interface element or 
object (step 240). 

Fig. 6 illustrates the operation of the map component 104. In the process 250, the 
map component 104 instructs the fashion component 108 to present itself (step 252). Next, 
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the map component 104 listens for events generated during the presentation or by the fashion 
component 108 (step 254). The logic of the application is then executed (step 256). 

Fig. 7A shows pseudo-code associated with an exemplary user interface "next." The 
"next" user interface includes a plurality of maps: Mapl-Map3. Map 1 is associated with 
fashion 1 which defines a user interface region called top. Map 2 is associated with a fashion 
component 2 and is associated with a right region. Finally, Map 3 is associated with fashion 
component 3 and is used to define a left region. 

Pseudo-code associated with the user interface of Fig. 2 is shown in Fig. 7B. In this 
figure, the layout of the user interface is defined as having a top region such as the region 
120 of Fig. 2 as a vertical region. The vertical region is further specified as having a right 
region such as the region 124 of Fig. 2 and a left region such as the region 122 of Fig. 2, both 
of which are laid horizontally adjacent each other under the top region. 

Fig. 8 is a diagram 300 illustrating the operation of one implementation embodying 
the map and fashion components. First, a user interface is created (step 301). The user 
interface includes a map component 304, a fashion component 306 and one or more 
properties files 308. Before the user interface is executed, a binding process 302 combines 
the map fashion and property files into an executable ready form. During operation, the 
process 300 uses the fashion components to interpret the property files (step 310). 
Additionally, the map components can be used to interpret various selection events generated 
by the user (step 312). Further, the map components can be used to interpret the property 
files (step 316). 

During operation, the application can query a database, which can be resident with a 
host application or can be accessed remotely. In response, the database can provide 
information indicating the availability of different map components and fashion components. 
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In one implementation, the identification of a server that will handle the query is provided in 
the query. Based on the response, an initial selection of the map component 104 and the 
fashion component 108 is made based on the query response. The selection can also be 
based on locale, which customizes the selection base on the geographic location of the user. 
5 Based on the response to the query, an initial choice is made according to the 

environment variables and established preferences. A initial user interface is then presented 
to the user. The user can make certain selections using the UI defined in the map component 
or fashion component, and the presented interface can then be altered accordingly. 
Additionally, program logic embedded in the map component 104 can dictate the change of 
□ 10 the map component 104 or the fashion component 108, respectively. 
+; Fig. 9 shows a process 320 that is executed in one embodiment of the invention is 

, shown. First, an application program is initiated (step 322). The execution of the program 

rj generates one or more queries directed at a database enquiring as to the availability of maps, 

i-* fashions and locale (step 324). Next, a responsive database generates various environment 

M> 15 variables and preferences, which are sent back to the application program (step 326). Next, 
'O the application program selects a map component (step 328). The application program then 

selects and loads a fashion component (step 330). Finally, a locale is selected and loaded 
(step 332). Next the user interface is created user (step 324) and presented to the user (step 
336). During this presentation, the fashion component uses the locale to localize data 
20 presented to the user (step 338). Further, the program and/or business logic information 
contained in the map component is used to either change the map component itself or the 
fashion component (step 340). From step 336, the user can choose anothe r map, fashion or 
locale (stgp.342). 
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Fig. 10 shows aparatus associated with the process of Fig. 9. First, a remote database 
containing map components, fashion components and locale information is connected to a 
network 410. In addition to the map, fashion and locale information in the database 402, a 
property file can be contained in a second database 404 that is connected to the network 410. 
5 On the other side of the network, the request is handled by application software, 

operating system interfaces and device drivers 422. The application, operating system 
interfaces. and device drivers receive user instructions, as captured by an input/output 
hardware interface 424. The hardware 424 in turn drives a display device 426 and receives 
inputs from a user input device 428. Based on the map component, fashion component and 

10 locale information, an initial user interface is generated. The user interface can then be 
presented based on the user input as provided to the user input device 428. For each 
presentation, the map and the fashion can be changed independently of the other during 
execution of a program. 

As shown above, the logic and appearance of a user interface can be built 

1 5 independently. Hence, an application visual designer can create the user interface without 
incurring delays associated with involving a programmer. Moreover, the UI specification is 
declarative rather than procedural Thus, high-level specifications can be synthesized 
without a detailed procedural description of the required action. The appearances can be 
presented using a graphical interface, or can be presented in a non- visual modality such as a 

20 voice user interface that can be used in speech driven computers, including telephones and 
voice activated computers. Moreover, much of the UI is exposed in a way that facilitates 
localization by presenting localized text, images, and other user interface elements. 

The techniques described here may be implemented in hardware or software, or a 
combination of the two. Preferably, the techniques are implemented in computer programs 
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executing on programmable computers that each includes a processor, a storage medium 
readable by the processor (including volatile and nonvolatile memory and/or storage 
elements), and suitable input and output devices. Program code is applied to data entered 
using an input device to perform the functions described and to generate output information. 
5 The output information is applied to one or more output devices. 

Figure 1 1 illustrates one such computer system 600, including a CPU 610, a RAM 
620, and an I/O controller 630 coupled by a CPU bus 640. The I/O controller 630 is also 
coupled by an I/O bus 650 to input devices such as a keyboard 660 and a mouse 670, and 
output devices such as a monitor 680. Variations are within the scope of the following 

10 claims. For example, instead of using a mouse as the input devices, a pressure-sensitive pen 
or tablet may be used to generate the cursor position information. 

Moreover, each program is preferably implemented in a high level procedural or 
object-oriented programming language to communicate with a computer system. However, 
the programs can be implemented in assembly or machine language, if desired. In any case, 

1 5 the language may be a compiled or interpreted language. Each such computer program is 
preferably stored on a storage medium or device (e.g., CD-ROM, hard disk or magnetic 
diskette) that is readable by a general or special purpose programmable computer for 
configuring and operating the computer when the storage medium or device is read by the 
computer to perform the procedures described. The system also may be implemented as a 

20 computer-readable storage medium, configured with a computer program, where the storage 
medium so configured causes a computer to operate in a specific and predefined manner. 
While the invention has been shown and described with reference to an embodiment thereof, 
those skilled in the art will understand that the above and other changes in form and detail 
may be made without departing from the spirit and scope of the following claims. 
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